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, ^ ■ Abstract 

■ We study a class of Markovian systems of elements taking values in [0, 1] that 
CS| ' evolve in discrete time t via randomized replacement rules based on the ranks of the 

' elements. These rank-driven processes are inspired by variants of the Bak-Sneppen 

' model of evolution, in which the system represents an evolutionary 'fitness landscape' 

■ and which is famous as a simple model displaying self-organized criticality. Our main 
r~| ! results are concerned with long-time large-A^ asymptotics for the general model in 

I which, at each time step, K randomly chosen elements are discarded and replaced by 

^ ■ independent U[0, 1] variables, where the ranks of the elements to be replaced are chosen, 

I— I, independently at each time step, according to a distribution on {1,2,... , N}^ . 

I I Our main results are that, under appropriate conditions on kjv, the system exhibits 

^ ' threshold behaviour at s* € [0,1], where s* is a function of kn, and the marginal 

. distribution of a randomly selected element converges to U[s* , 1] as t — )■ co and N oo. 

I Of this class of models, results in the literature have previously been given for special 

Tlj- ' cases only, namely the 'mean-field' or 'random neighbour' Bak-Sneppen model. Our 

. proofs avoid the heuristic arguments of some of the previous work and use Foster- 

I Lyapunov ideas. Our results extend existing results and establish their natural, more 

' general context. We derive some more specialized results for the particular case where 

^ ■ K = 2. One of our technical tools is a result on convergence of stationary distributions 

• ^ \ for families of uniformly ergodic Markov chains on increasing state-spaces, which may 

^ ' be of independent interest. 



Keywords: Bak-Sneppen evolution model; self-organized criticality; Markov process on order 
statistics; phase transition; interacting particle system. 

AMS 2010 Subject Classifications: 60J05 (Primary) 60J10, 60K35, 82B26, 92D15 (Second- 
ary) 

1 Introduction 

Bak and Sneppen [5] introduced a simple stochastic model of evolution which initiated a 
considerable body of research by physicists and mathematicians. The Bak-Sneppen model 
has proved so influential because it is simple to describe and not difficult to simulate, and, 
while being challenging to analyse rigorously, demonstrates highly non-trivial behaviour: it 
is said to exhibit 'self-organized criticality' (see e.g. [20]). 
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The Bak-Sneppen model is as follows. Consider the sites 1, 2, . . . , arranged cyclically, 
so that site k has neighbours k — 1 and k + 1 (working modulo A^). Each site, corresponding 
to a species in the model, is initially assigned an independent t/[0, 1] random variable repres- 
enting a 'fitness' value for the species; here and subsequently U[a, b] stands for the uniform 
distribution on the interval [a,b]. The Bak-Sneppen model is a discrete-time Markov pro- 
cess, where at each step the minimal fitness value and the values at the two neighbouring 
sites are replaced by three independent U[0, 1] random variables. A variation on the model 
is the (maximal) anisotropic Bak-Sneppen model [TH] in which, at each step, only the right 
neighbour of the site with minimal fitness is updated along with the minimal value. 

A large physics literature is devoted to these models. Simulations suggest that the equi- 
librium distribution of the fitness at any particular site approaches U[s*, 1] in the N oo 
limit, for some threshold value s*; simulations give s* ~ 0.667 for the original Bak-Sneppen 
model and s* ~ 0.724 for the anisotropic model [13]. There is a much smaller number of 
mathematical papers on the model and its variants: see e.g. [l5l[T6l[23l|2l]; see also the 
thesis [H]. It is a challenge to obtain further rigorous results for such models. 

A simpler model can be formulated by removing the underlying topology, and such 'mean 
field' or 'random neighbour' versions of the model have also received attention in the liter- 
ature: see e.g. [3 El [121 EDI Ell 121] • For example, the mean-field version of the anisotropic 
Bak-Sneppen model again has A^ sites each endowed with a fitness in [0, 1]. At each step, 
the minimal fitness is replaced, along with one other fitness chosen uniformly at random from 
the remaining A^ — 1 sites. Again the replacement fitnesses are independent f/[0, 1] variables. 

Such mean-field models display some features qualitatively similar to the original Bak- 
Sneppen model, but give little indication of how the distinctive asymptotics of the Bak- 
Sneppen model, in which the topology plays a key role, might arise. In particular, changing 
the topology of the model changes the value of the threshold s* in a way that the mean-field 
models cannot account for. In the present paper we study some generalizations of the mean- 
field model described informally above, which we call rank-driven processes. These models 
represent one possibility for showing how the influence of topology might be replicated by 
simpler features. 

In these more general models, we again have A^ sites, and at each time step some fixed 
number of fitness values are selected for replacement, but for this selection process we allow 
general stochastic rules based on the ranks of the values. These rank-driven processes are 
Markov processes on ranked sequences, or order statistics (see Sections H] and [3] for formal 
definitions). 

To give a concrete example, we could, at each step, replace the minimal fitness along with 
the Rth ranked value, where R is chosen independently at each step from some distribution 
on {2, . . . , A^}, with R = 2 corresponding to the second smallest value, and so on. This 
model generalizes that of [7], which has R uniform on {2, . . . , A^}, and exhibits much richer 
behaviour. Specifically, the threshold s* depends explicitly on the distribution chosen for R: 
in this way, the distribution of R is playing a role analogous to the underlying topology in 
the Bak-Sneppen models. 

Such rank-driven processes are of interest in their own right, but our motivation for 
studying them also arises from an attempt to understand the original Bak-Sneppen model, 
where the topology plays a key role. While the Bak-Sneppen model can be viewed as a 
Markov process on the space [0, 1]^, it gives rise to a decidedly non-Markovian process on 
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order statistics. At the same time, as we explore in detail in [T7] , there is an algorithmic way 
to associate to the Bak-Sneppen model a rank-driven process (a process that is Markovian on 
order statistics) and which, according to numerical evidence, shares a number of important 
properties with the Bak-Sneppen model. The aim of the present paper is to present a 
rigorous analysis of rank-driven processes. 

The outline of the remainder of the paper is as follows. In Section [2] we discuss an 
introductory example in which a single value is updated at each step. In Section [3] we 
describe the general rank-driven processes that we consider and state our main theorems on 
asymptotic behaviour. In Section H] we focus on a specific class of examples, generalizing the 
mean-field anisotropic Bak-Sneppen model (TJIIlj, and give some more detailed results. We 
emphasize the difference in nature of the results in Sections [3] and HI in the former, the results 
cover a very general class of processes and the proofs are robust, using Foster-Lyapunov 
arguments and general theory of Markov processes, while in the latter, we specialize to a 
narrower class of processes and exploit their special structure. It is likely that analogues 
of our results from Section H] could be obtained for other processes, but the details would 
depend on the particular processes studied. In Section |5] we make some further remarks and 
state some open problems. Section is devoted to the proofs of the main results in Section 
ini while Section [7] is devoted to the proofs of the results in Section |H The Appendix, Section 
m gives one of our key technical tools on the asymptotics of families of Markov chains that 
are uniformly ergo die in a precise sense. 

2 Warm-up example: Replace the A:th-ranked value 

We start by describing a particularly simple model, which can be solved completely, to 
demonstrate some ideas that will be useful in greater generality later on. It will be convenient 
to view all of our models as Markov chains on ranked sequences, or order statistics. Fix 
N E N := {1, 2, . . .}. Given a vector (x^, x^, . . . , x^) we write the corresponding increasing 
order statistics as 

(x(i\x(2),...,x(^))=ord(x\...,a;^), 
where x^^"^ < x*-^^ < ■ ■ ■ < x^'^K Let A^r denote the 'simplex' 

Ajv := {{x\ . . . , x^) G [0, 1]^ : < < ■ ■ ■ < x^}. 

We study stochastic processes on A^r indexed by discrete time := {0, 1, 2, . . .}. 

Let Ui,U2,-- - denote a sequence of independent ?7[0, 1] random variables. Define a 
Markov process Xt on An with transition rule such that, given Xt, 

= ord{[/,+i, ), \ . . . , Xf )}; 

in other words, at each step, discard the smallest value and replace it by a U[0, 1] random 
variable. To make clear the dependence on the model parameter N, we write for the 
probability measure associated with this model and Eat for the corresponding expectation. 

It is natural to anticipate that Xt should approach (as t — oo) a limiting (stationary) 
distribution; we show in this section that this is indeed the case. Assuming such a stationary 
distribution exists, and is unique, we can guess what it must be: the distribution of the 
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random vector (f/, 1,1,1,...,!) (a f/[0, 1] variable followed by — 1 units) is invariant under 
the evolution of the Markov chain. The process itself lives on a relatively complicated 
state-space, and at first sight it might seem that some fairly sophisticated argument would 
be needed to show that it has a unique stationary distribution. In fact, we can reduce the 
problem to a simpler problem on a finite state-space as follows. 
For each s G [0, 1], define the counting function 

N 

(.) := #{z e {1, 2, . . . , iV} : < = E ^i^^ ^ (2.1) 

i=l 

i.e., C^{s) is the number of values of magnitude at most s in the system at time t. (Here 
and throughout we use to denote the number of elements of a finite set A.) Then, 
for a fixed t, Xt is characterized by the counting functions (C/^(s))sg[o,i]. For a specific s, 
{s) encodes marginal information about the xf'\ since the two events {Cf^ {s) > k} and 
{Xjf*''* < s} are equivalent. By an analysis of the auxiliary stochastic processes C^{s) we 
will prove the following result, which deals with the more general model in which, at each 
time step, the point with rank k is replaced. 

Proposition 2.1. Let N & N and k G {1,2, . . . , A^}. For the model in which at each step 
we replace the kth-ranked value by an independent U[0, 1] value, we have that, as t ^ oo, 

(x«,xf),. ..,xf)) 4(0,.. .,0,^,1,. ..,1), 

where U , the kth coordinate of the limit vector, has a f/[0, 1] distribution. 

li k = k{N) is such that k{N)/N — > 6' G [0, 1] as — > oo, a consequence of Proposition 
12.11 is that the distribution of a uniformly chosen point converges (as t — )■ oo and then 
— )■ oo) to the distribution with an atom of mass 6' at and an atom of mass 1 — ^ at 1. 
For example, if we always replace a median value, 6 = 1/2 and the limit distribution has two 
atoms of size 1/2 at and 1. 

Proof of Proposition \2. 1[ It is not hard to see that C^{s) is a Markov chain on 
{0, 1,2,..., A^}. The transition probabilities p%{n, m) := FN[Cf^i{s) = m \ {s) = n] are 
given for n G {0, . . . , — 1} by p%{n, n) = 1 — s and p%{n,n + l) = s, and for n G {k, . . . , A^} 
by p%{n^ n) = s and p%{n, n — 1) = 1 — s. For s G (0, 1) the Markov chain is reducible and 
has a single recurrent class consisting of the states k — 1 and k. It is easy to compute the 
stationary distribution and for s G (0, 1) we obtain, 

{1 — s ifn = fc — 1 
s iin = k 

lin^ {k-l,k} 

by standard Markov chain theory. In particular, for s G (0, 1), 

lim P^[A:f ^ <s\ = lim ¥n[C^{s) >k]=s. 

t—^oo t—^oo 
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That is, Xr converges in distribution to a f/[0, 1] variable. Moreover, if A; > 1, for any 
s e (0,1), P[xf"^^ < s] = P[Cf (s) > A; - 1] -> 1, which imphes that X^''"^^ converges 
in probabihty to zero. Similarly, if A; < A^, for any s G (0, 1), P[xf < s] = P[Cf (s) > 
/c + 1] — 0, which implies that xj:'''^^^ converges in probability to 1. Thus we have proved 
the marginal result that, as t — oo, for ?7 a f/[0, 1] random variable, 

^0, (Kk), ) ^ [/, ^1, {i>k), 

in distribution. Then the Cramer-Wold device (convergence in distribution of an X- 
dimensional random vector is implied by convergence in distribution of all linear combina- 
tions of its components: see e.g. [10], p. 147]) together with Slutsky's theorem (if Yn converges 
in distribution to a random limit Y and Z„ converges in distribution to a deterministic limit 
z, then Yn + converges in distribution toY \ z: see e.g. [TUl p. 72]) enable us to deduce 
the joint convergence. □ 

Remark 2.1. This simple example shows special features that will not recur in the general 
case, (i) Here we obtained a result for any fixed N > 1; in the general case, we will typically 
state results as N ^ oo. (ii) Since all hut one of the X^f* had a degenerate limit distribu- 
tion, we were able to use a soft argument to deduce convergence of the joint distribution of 
{x[^\ . . . ,Xj^'*) from the convergence of the marginal distributions. 



3 Rank-driven processes and threshold behaviour 

In this section we give a general definition of a rank- driven process and present some 
fundamental results on its asymptotic properties. Fix X (the number of points) and 
X e {1, . . . , X} (the number of replacements at each step). Define the set 

X^:={1,2,...,X}^. 

The model will be specified by a selection distribution. Let denote a random X-vector 
with distinct components in {1, . . . , X}. In components, write = [R^ , ■ ■ ■ , Rk)- We 
suppose that R^ is exchangeable, i.e., its distribution is invariant under permutations of 
its components. The distribution of i?^ can be described by a probability mass function 
ktv : X^ —J- [0,1] that is symmetric under permutations of its arguments, so Fn[R^ = 
ii, . . . , i?^ = ix] = HN{ii, ■■■,iK)- 

We define a Markov chain {Xt)t(zz+ of ranked sequences Xt = (X^^-*, . . . , X^^^-*). The 
initial distribution Xq can be arbitrary. The randomness of the process will be introduced 
via independent t/[0, 1] random variables Ui,U2,-.. and independent copies of R^ , which 
we denote by i?^(l), i?^(2), .... In components, write R^{t) = (i?f (t), . . . , i?^(t)). The 
transition law of the Markov chain is as follows. 

Given Xf , discard the elements of (distinct) ranks specified by Ri{t + 1), . . . , R^it + 1) 
and replace them by K new independent f/[0, 1] random variables, namely Uxt+i, ■ ■ ■ Uki+k] 
then rank the new sequence. That is, we take X^+i to be 

ora I ,...,Ai ^UKt+l.y^t 'J Kt+K.y^t ■,■■■■,^1 )■ 
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Note that ties are permitted. 

For i G N let 5'7v(i) := f']\r[Rf = i], the marginal distribution of a specific component of 
i?^. Denote the corresponding distribution function by 

n n N N 

GN{n) := Fn[R^ <n] = J2 SnU) = 5Z 5Z ' ' ' ^^^'^^ '2, • • • , ir)- (3.1) 

i=l ii=l 42=1 *K=1 

We make some further assumptions on the selection distribution. Assumption (Al) will 
ensure that an irreducibility property holds, excluding some degenerate cases, while (A2) 
regulates the N ^ 00 behaviour of the selection rule. 

(Al) li K = 1, suppose that gN{i) > for all i G {1,...,A^}. U K > 2, suppose that 

QNil) > 0. 

(A2) Suppose that for any k < K, for all distinct ii, . . . ,1^ G N, the limit ^{ii, . . . ,ik) '■= 
limAT^oo Fn[Ri = k, ■ ■ ■ ,Rk = ik] exists. 

Note that the limits in (A2) need not constitute proper distributions on N'^: there may 
be some loss of mass. Indeed, the possibility of a defective distribution as the limit in (A2) 
plays a central role in the asymptotics of the rank-driven process, as we shall describe below. 
A proper distribution can be recovered on (N U {oo})-^ by correctly accounting for the lost 
mass, and then (A2) can be interpreted as saying that i?^ converges in distribution to a 
random vector on (N U {oo})-^: see Section for details. A consequence of (A2) is that 

lim QNin) = g{n) and lim GN{n) = G{n) (3.2) 

exist for all n E N; then G is a (possibly defective) distribution function on N. (Note that 
g{i) = Given (A2), we make an assumption on g analogous to (Al): 

(A3) If = 1, suppose that g{i) > for all i eN. If K > 2, suppose that g{l) > 0. 

We will show that a crucial parameter for the asymptotics of the process is 

s* := lim G{n) = lim lim Gn^u). (3.3) 

If (A2) holds, then the A^-limit exists, and s* G [0, 1] is well-defined. The value of s* captures 
the 'asymptotic atomicity' of Gtv in a certain sense. 

Before stating our first results, we describe some concrete examples. For specifying our 
examples, it is more convenient to work with a version of ka? on ranked sequences, namely 
7Ar defined foi ii < ■ ■ ■ < ix by ■jNih, • • • , iK) = KlK^^ii, . . . , ix)- With this notation, note 
that 

9Nii) = ^ I 5Z 77v(^,^2,«3, • • • ,«A') + 7N{i2,i,i3,---,iK) 

\i<i2<h<--<'i'K i2<i<i3<---<iK 

+ ■■■+ Yl 7Nii2,i3,- ■ ■ ,iK,i)\ , (3.4) 

«2<«3<---<«/f<i / 

where each sum is over i2,i3, . . . ,iK ^ {1? • • • j N} satisfying the given rank constraints. 

We describe three examples, by giving the non-zero values of either 77V or kj^, as con- 
venient; (El) was discussed in Section [21 while we study examples (E2) and (E3) in detail 
in Section HI 
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Example (El). Take K = 1 and 7Ar(/c) = 1, i.e., replace the kth ranked element only 
each time. In this case gN{k) = 1 as well. 



Example (E2). Let K = 2 and 7Ar(l,j) = for j G {2, ...,7V}, i.e., each time we 
replace the minimal element and one other uniformly chosen point. This model has been 
studied by and others. From (13. 4p we have that in this case 5'7v(l) = 1/2 and, for 
z G {2, . . . , iV}, ^7v(0 = 2(F3T) • Moreover, ^(1) = 1/2 and g{i) = for z ^ 1. 

Example (E3). (A generalization of (E2).) Let K > 2 and let (pN be a symmetric 
probability mass function on X^~^. Set KAr(l, i2, • • • , ^k) = -^~"^0Af(^25 • • • , ix)- So now we 
replace the minimal element and K — 1 other randomly chosen points, where the distri- 
bution on the 'other' points is given by (pjy. Then 5'Ar(l) = 1/K and, for i G {2, . . . , A^}, 
9Nii) = ^fNii) where f^ii) = ^^ii^is, ■ ■ ■ ^ix)- Write FN{n) = Er=i/Af(^)- 

Assume that F{n) = limAr_^oo Fiy{n) exists for all n, and set a = lim„_^oo F{n) G [0, 1]. 

The assumptions (Al) and (A3) are satisfied by (E2) and (E3), but not (El), while (A2) 
is satisfied by (El), (E2), and (E3). 

We will work with the counting functions defined by (12. ip . Our first main result. Theorem 
13. II below, demonstrates a phase transition in the asymptotic behaviour of the system at the 
threshold value s = s*; note that part of the theorem is the non-trivial statement that 
lim7v-s.oo hm(_^oo EAr[C/^(s)] exists in [0, oo]. 

Theorem 3.1. Suppose that (Al), (A2), and (A3) hold, so that s* given by Ii3.3\) exists in 
[0,1]. Then 

hm limE^[Cf(s)]|<°^ (3.5) 

N^oo t^oo I = OO %J S > S 

Theorem 13.11 shows that s* is a threshold value for the model in the sense that 
s* = sup{s > : lim lim En[C^{s)] < oo} 

Af— s-oo !>oo 

is well defined (with the convention sup0 = 0). Example (El) has s* = 1, although Theorem 
13.11 does not apply directly (since (Al) and (A3) fail). Example (E2) has s* = 1/2, while 
Example (E3) has s* = ^{1 + {K - l)a). 

For the next result we assume that the distribution Gn given by (13. ip is 'eventually 
uniform' in a certain sense; roughly speaking we will suppose that gN{n) ~ for n large 
enough. The precise condition that we will use is as follows. 

(A4) Suppose that there exists uq G {2, 3, . . .} such that 

NiG^in) - s*) , 



lim sup 

^-*oo no<n<N 



n — Uq + 1 



0. 



For instance. Example (E2) satisfies condition (A4) with s* = 1/2 and no = 2, since G^^n) = 
\ + 2(N-i) ■ Example (E3), GAr(n) = + ^FN{n), so that condition (A4) holds if FAr(n) 
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satisfies a similar condition, namely 



lim sup 



7V-J-00 



no<n<N 



N{Fr,{n)-a) 

- [l — a, 



n — Uq + 1 



0. (3.6) 



Our next result shows the threshold phenomenon at the 'O(A^)' scale. We can define a 
threshold parameter 

s* := sup{s > : lim lim (N-^En\C^ (s)]) = 0}. (3.7) 

If both s* and s*, as given by (13.31) and (13.71) respectively, are well-defined, then clearly 
s* < s*. Theorem 13.21 shows that, under assumption (A4), s* = s*; in other words, the 
transition is sharp. We use the notation 

Theorem 3.2. Suppose that (Al), (A2), (A3), and (A4) hold. With V{s) as given by l[37^) . 
we have that for any s G [0, 1], 

Un, , to (MfMU (3.9) 



Another way to interpret Theorem 13.21 is as follows. Let denote X^^^ where M is a 
random variable with Pjv[M = j] = 1/N for j G {1, . . . , A^}. Then 

N N 

^n[X: <s] = N-^J2f[xP <s] = AT-^E^ l{xf) <s} = N-^En[C^{s)], 

i=l 1=1 

by (12.11) . so that the conclusion of Theorem 13.21 is equivalent to, for s G [0, 1], 

lim lim Fn[X* < s] = V{s); 

in other words, the marginal distribution of a 'typical' point converges (as t — ?■ oo then 
A^ — 7- oo) to a f/[s*,l] distribution. Note that some condition along the lines of (A4) is 
needed for this result to hold: see the example in Remark 13.21 below. 

Remark 3.1. In this paper we restrict attention to the case where the distribution of replace- 
ments is U[0, 1], but instead of Ui, U2, ■ ■ ■ one could take independent copies of a nonnegative 
random variable W with distribution function p. The results with the U[0, 1] distribution im- 
mediately generalize to distributions p that are continuous, supported on a single interval, and 
strictly increasing on that interval: to see this, note that (xi, . . . ,Xn) H- (p(xi), . . . ,p{xisf)) 
preserves ranks and p{W) has a f/[0, 1] distribution, so that the dynamics of the process are 
preserved, up to the change of scale s H- p(s). Thus our results immediately extend to this 
class of distributions W . 
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Remark 3.2. Our results can he translated into complementary results by reversing the 
ranking and looking at N — C/^(s). Indeed, N — counts the number of points in 

(s, 1]; translating Theorem VJ. 1\ shows that under appropriate versions of (Al)-(A3) (in which 
conditions on gN{n) are replaced by conditions on gN{N — n + 1)) the threshold 

= mi{s < 1 : lim lim En[N - Cf (s)] < oo} 

N^oo t—^oo 

is given by 

= lim lim Gi\i{N — n). 

For example, suppose that K = 2 and we always replace the smallest and the largest points. 
I.e., gNil) = gNiN) = 1/2. Then Givin) = (1 + l{n = N})/2, so that G{n) = 1/2 and 
s* = 1/2. But also, s* = 1/2. So the expected number of points in any interval not containing 
1/2 remains finite as N oo; in other words, the marginal distribution of a typical point 
converges to a unit point mass at 1/2. This example also serves to demonstrate that Theorem 
\3.2\ may fail if (A4) does not hold. 

4 Detailed example: Replace the minimum and one 
other 

In this section we present some more specific results to complement our general results from 
Section [3l To do so, we specialize to the K = 2 case of Example (E3) from Section [3l in 
which we replace the smallest value and choose the other value to replace from {2, . . . , A^} 
according to a probability distribution f^. Write 

k 

FN{k):=J2fN{j), (4.1) 
i=2 

for the corresponding distribution function, adopting the usual convention that an empty 
sum is zero, so that -F/v(0) = -F/v(l) = 0. 

In the general set-up of Section [3l we have Kisf{l,i) = /7v(i)/2, 5'Ar(l) = 1/2 and gN^i) = 
fM{i)/2 for i e {2,...,A^}. Here, assumption (Al) and (A3) are automatically satisfied; 
Assumption (A2) becomes a condition on f^ (or -F/v), namely that 

lim F^(n) = F{n) (4.2) 

AT— >oo 

exists for all n>2. The present version of (A2) is then: 
(A2') Suppose that holds. 
Under (A2'), 

a := lim F{n) (4.3) 

n—^oo 

exists in [0,1]. Indeed, since GN{n) = (1 + FN{n))/2, we have that (A2') implies that s* 
given by (13. 3p satisfies s* = 
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Before stating our results, we comment briefly on their relation to previous work in the 
literature. The model of this section includes that studied by de Boer et al. [7] amongst 
others (see e.g. [201 §5.2.5]); the model of [7] is the special case where Fjq{n) = jf^, which 
satisfies (A2) with a = 0. Thus the a = cases of our results are not surprising in view 
of the (not completely rigorous) arguments in [7], or the heuristic analysis in [20| §5.2.5] 
that neglects correlations between the X^''\ but our results are more general even in the 
case a = 0, and we show explicitly the dependence of the phase transition on Fn via the 
parameter a. Moreover, one aim of the present work is to give a more rigorous approach to 
the results of [7] in the present considerably more general setting. 

In this setting, the following result is immediate from Theorem 13.11 

Theorem 4.1. Suppose that (A2') holds. Then 



< oo if s < ^ 

TV-i-ooi^oo * ^ I = cx) if S > 2 



limlimE^[Cf(.)] _ yZil- (4.4) 



Similarly, we have the following translation of Theorem 13.21 into this setting. The appro- 
priate version of condition (A4) is: 

(A4') Suppose that F/v satisfies fl3.6p . 

Theorem 4.2. Suppose that (A2! ) and (A4') hold. With V{s) as given by / TO]) . 

hm hm (^^^^^P^)=V{s), se[0,l]. 



N^oot^oo \ N 

Remark 4.1. // instead of a f/[0, 1] distribution we use a distribution p for replacement 
points, as described in Remark \ 3.1[ then the threshold exhibited in Theorem \4.1\ becomes 
s* = p~^{^-^); the inverse is well-defined when p satisfies the conditions described in 
Remark{3Jl 

Now we move on to our detailed results concerning the case a = 0; note that a = if and 
only if /Ar(n) -> as — oo for any n. The case a = includes the discrete uniform case 
(as considered in [7]) in which fNiji) = jfZi^ but includes many other possibilities. Theorem 
14.11 shows that when a = the phase transition occurs at s* = 1/2. The next result gives 
more information, giving an explicit expression for the limiting equilibrium expectation in 
the case in which it is finite. 

Theorem 4.3. Suppose that (A2') holds and that a = 0. Then 

hm limE^[Cf(s)] = |'^+^ r^W^'^'- (4-^) 

N^oct-i-oo loo ifs>l/2 

We also have the following explicit description of the limit distribution. 
Theorem 4.4. Suppose that (A2') holds and that a = 0. If s < 1/2, then for any n G Z'^, 

lim lim P7v[Cf (s) = n] = 7r"(n), 

N^oo t—^oo 
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where 

7r"(0) = 1 - 2s; 
TT'd) = 2s- 



s 



1 - s 



On the other hand, if s > 1/2, then for any n G liiiiAr^oo limt-^oo P7v[C*j^('5) < = 0. 

Remark 4.2. (^z^ T/ie corresponding stationary probabilities put forward in ^ do not sum to 
1 (see equations (10)-(12) in Our argument is similar (based on the use of the 'N = oo' 
Markov chain) but we try to give a fuller justification, (ii) Let 

Tn{s) := min{t G N : Cf (s) = 0}. (4.7) 

By standard Markov chain theory, n%{n) = {Em[tn{s) \ Cq{s) = 0])"^. So an immediate 
consequence of Theorem 4-4 is that (cf equation (16) of ^) 

tfs< 1/2 



lim E^[r^(s) I C^{s) = 0] = 

N^oo I (X) if S > 1/2 

We also prove explicit limiting (marginal) distributions for the lower order statistics 
themselves. We use the notation 



^nis) , ,2(n-l) ^ ^ • (4.8) 



2s if n = 1 

Theorem 4.5. Suppose that (A2') holds and that a = 0. Then for n G N, 



lim lim P7v[xi"^ < s] = <( /i„(s) if0<s<l/2. (4.9) 

ifs>l/2 

The n = 1 case of (14. 9 p says that the large A^, long-time distribution of the smallest 
component approaches a U[0, 1/2] distribution. The distributions arising for n > 2 are not 
so standard, but, as n — )■ oo, they approach a unit point mass at 1/2. 

Also note that Theorem 14.51 yields convergence of moments of the Xf'^\ For example, for 
any keN, lim7v->oo limt^oo E7v[(xf V] = 2~''/{k + 1), and for n > 2 and any keN, 

lim lim EAr[(XrT] = 2"*^ : 2Fi{2n - 2,2n + k - 2]2n + k - l]l/2). (4.10) 

N^oot^oo 2n -\- k — 2 

To see this, note that since X^"'* is uniformly bounded, its moments converge to those of the 
distribution by bounded convergence, so we have 

lim lim Ejv[(xi"^)^] = k / s^^^ ( lim lim P7v[xf ^ > s]] ds 

AT— >oo t— >oo Jq VA''— >oo f— >-oo / 
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z/s < 



= k s''^\l - hn{s))ds. 
Jo 

When n = 1 this is k /q^^^(1 — 2s)s^~^ds which yields the claimed result. When n> 2, using 
the substitution u = 2s, the limit becomes 

Jo 

which gives fl4.10p via the integral representation of the hypergeometric function. 

5 Further remarks and open problems 

A multidimensional model 

Allowing more general distributions W, as described in Remark 13. enables some multi- 
dimensional models to fit within the scope of our results. We describe one example. Let Z 
be a uniform random vector on [0, 1]^, and let || • || denote the Euclidean norm. Starting 
with N points in [0, 1]^, iterate the following Markovian model: at each step in discrete 
time, replace the minimal-ranked point, where the ranking is in order of increasing Euclidean 
distance from the origin, and another point (chosen uniformly at random) with independent 
copies of Z. This model corresponds to the model described in Section H] but with the 
Ui replaced by copies of W = \\Z\\, and with a = 0. Elementary calculations show that 

2 

p{x) := F[W < x] = ^ for x G [0,1] {p{x) is more complicated for x > 1), so that the 
phase transition (see Remark l^?T|) occurs at s* = p~^(l/2) = \/2/'k ^ 0.80. See Figure[T]for 
a simulation. 

A partial-order-driven process 

Here is a variation on the multidimensional model of the previous example governed by 
a partial order rather than a total order. Again consider a system of N points in [0, 1]^. 
Consider the co-ordinatewise partial order '=^' under which {xi,yi) ^ (2:2, 1/2) if and only if 
Xi < X2 and yi < y2', a point x of a finite set X C [0, 1]^ is minimal if and only if there 
is no y G A" \ {x} for which y ^ x. Now define a discrete-time Markov process as follows: 
at each step, replace a minimal element of the N points (chosen uniformly at random from 
amongst all possibilities) and a non-minimal element (again, chosen uniformly at random); 
all new points are independent and uniform on [0, 1]^. This model seems more difficult to 
study than the previous one, although simulations suggest qualitatively similar asymptotic 
behaviour: see Figure [2l 

A repeated beauty contest 

We describe a process of a different flavour to those previously considered, in which the 
update rule depends not only on the ranks of the points; this is a variation on a Keynesian 
beauty contest. 
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Figure 1: Simulation of the model in which, at each step, the closest point to the origin and 
one uniformly random other point are replaced by independent uniform random points on 
[0, 1]^, with = 10^ points and t = 10^ steps. The initial distribution was independent 
uniform points on [0, 1]^. Also shown in the figure is part of the circle centred at the origin 
with radius \/2/7r. 

Fix a parameter p > 0. Start with a uniform array of A^ elements on [0,1]. At each 
step, compute the mean fi of the A^ elements, and replace by a f/[0, 1] random variable the 
element that is farthest (amongst all the A^ points) from pfi. Thus at each step, either the 
minimum or maximum is replaced, depending on the current configuration. 

This is related to the "p-beauty contest" [271 P- 72] in which A^ players choose a number 
between and 100, the winner being the player whose choice is closest to p times the average 
of all the A^ choices. The stochastic process described above is a repeated, randomized version 
of this game (without any learning, and with random player behaviour) in which the worst 
performer is replaced by a new player. 

According to simulations and heuristic considerations, the equilibrium distribution of a 
typical point approaches, for large A^, a point mass at (1) in the case p < 1 {p > 1). The 
case p = 1 is more subtle, and is reminiscent of a Polya urn. Stochastic approximation ideas 
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Figure 2: Simulation of the model in which, at each step, one ^-minimal element and one 
non-minimal element (each uniformly chosen) are replaced by independent uniform random 
points on [0, 1]^, with = 10^ points and t = 10^ steps. The initial distribution was 
independent uniform points on [0, 1]^. Can the threshold curve be characterized? 



(see e.g. 



may be relevant in studying this model. 



6 General thresholds: Proofs for Section [3 



6.1 Overview 

This section contains the proofs of our general results from Section |3l and is arranged as 
follows. In Section [6.21 we give a basic result on the Markov chains C[^{s). To study the 
N oo asymptotics of these Markov chains, at least when s < s*, we introduce an W = oo' 
Markov chain Ct{s). In Section I6l3] we show that we can define Ct{s) in a consistent way, 
and we prove some of its basic properties. In Section [6^ we relate the asymptotic properties 
of the finite- chains C^{s) with s < s* to the chain Ct{s), making use of our technical 
results from Section [8l Then in Section 16.51 we complete the proofs of Theorems 13.11 and 13.21 
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6.2 The Markov chain (s) 

We have the following basic result. 

Lemma 6.1. Suppose that (Al) holds. Fix G N. Suppose that s G (0,1). Then C^{s) 
is an irreducible, aperiodic Markov chain on {0, 1, . . . , A^} with uniformly bounded jumps: 
^n[\C^i{s) — C^{s)\ > K] = 0. There exists a unique stationary distribution nlf, with 
n^in) > for all n G {0, 1, . . . , A^} and X]^=o ^Afl'^) ~ ■^'"^^ ^^^^ 

limP;v[Cf(s) =n] = 7r^(n), (6.1) 
for any initial distribution Cq{s). Moreover, 

N 

lim E^[Cf (s)] = y n7r^(n). (6.2) 

n=l 

Finally, with Gjy as defined at hS. the one-step mean drift of {s) is given by 

E^[Q^i(s) - Cf (s) I Cf (s) = n] = K{s - G^in)). (6.3) 

Note that the degenerate cases s G {0, 1} are excluded from Lemma [6TT} C/^(l) = a.s. 
for all t, while Cf (0) ^ db.S. clS t — )■ oo for any initial distribution C^(0). 

Proof of Lemma \6.1\ Note that 

{Cf (.) =n} = {Xi^) <.,..., Xi^) < >.,..., Xf) > 

the distribution of Q^i(s) depends only on {xl^\ . . . ,X^^^) through events of the form on 
the right-hand side of the last display. Specifically, given C/^(s) = n, we have that the 
increment Cl^-^^{s) — C^{s) is the number of the K new U[0, l]-distributed points that fall 
in [0, s] minus the number of the K points selected for removal whose rank was at most n. 
That is, with the notation 

Bt+i{s) := i^{te{Kt + l,...,Kt + K}:U,e [0, s]}, 
<i(^) := e {1, . . . , : Rfit + 1) < n}, 

we have that, given C^{s) = n, 

C,^,(.)=n + i?,+i(s)-<,(n). (6.4) 

Thus, given C/^(s) = n, the increment depends only on n and the variables R'^(t + 1), 
Uxt+i, ■ ■ ■ , Uxt+K, which are all independent of C/^(s). This demonstrates the Markov prop- 
erty. 

The bounded jumps property is clear by definition, and can also been seen from (16. 4p . 
To show irreducibility and aperiodicity, we show that 

P^[Q^i(s) = n I Cf (s) = n] > 0, (n G {0, 1, . . . , N}), 
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P^[Cm(^) =n + l\ Cf (s) = > 0, (n e {0, 1, . . . , iV - 1}), 
W4C,l,is) = n - 1 I Cf (s) = n] > 0, (n e {1, 2, . . . , N}). 

Since 5^+1 (s) and y4^;^(n) are independent given C/^(s) = n, it suffices to show that 
F^lBt+iis) = i] > for any i e {0, 1, . . . , /sT}, and that P^vf^i^iW = > for: (i) 
2 G {0, 1} if Ji" = 1; or (ii) i = 1 if K > 2. Then the intersection of two independent events 
of positive probabihty will yield any increment of Cj^(s) in { — 1,0,1}, as required. First 
consider Bt+i{s): this has a Bin{K, s) distribution, and so takes any value in {0, 1, . . . , K} 
with positive probability, provided s G (0, 1). Now consider Af_^^{n). Then 

n 

Klin) = Y.l{^e {<(t + 1), . . . , <(t + 1)}}. (6.5) 

4 = 1 

It follows from ([63]) that, for n > 1, A^i(n) > 1{1 G (t + 1), . . . , R^{t + 1)}}, so 

P^[<i(n) = 1] > P^[l G {R^{t + 1), . . . , <(t + 1)}] > P^[< = 1]. 

This latter probability is 5'Ar(l), which is positive by (Al). This completes the proof of 
irreducibility and aperiodicity in the case K > 2; it remains to show that ¥]y[Af^_^-^{n) = 0] > 
for n > when K = 1. Using the K = 1 case of (16. 5p . we obtain 

P^[Af^i(n) = 0] = ¥j^[R^{t + 1) > n] = 1 - G';v(n), 

by (13.11) . and 1 — GNiji) > since (Al) implies that in this case gN^i) > for some i > n. 

Thus the Markov chain is irreducible and aperiodic; it has a finite state-space, and so 
standard Markov chain theory implies the existence of a unique stationary distribution, for 
which ( 16. ip holds. Moreover, since Cf (s) is bounded by N, (O follows from (O). 

Finally we prove the statement (16. 3p . We take expectations in (16. 4p : Bt+i{s) has mean 
Ks, and taking expectations in (16. 5p we obtain 

n n 

nKiin)] = J2^[^e {<(t + 1), . . . , <(t + 1)}] = KJ2^[R^ = 
1=1 i=l 

by exchangeability. Thus from ( 13. ip we obtain ( 16. 3p . □ 

A key step in our analysis is to study the stationary distributions vr^ of the Markov chains 
Cl^{s), s G (0,1), whose existence is proved in Lemma l6.ll We consider irf^ as N ^ oo. 
One tool that we will use is a Markov chain Ct{s) on the whole of Z"*" that can be viewed in 
some sense as the N ^ oo limit of the Markov chains C^{s): this Markov chain we call the 
W = oo' chain, and we describe it in Section in Section [^31 we make precise the sense 
in which the W = oo' chain is a limit of the finite- chains. 

6.3 The 'N = oo' chain Ctis) 

Our asymptotic analysis makes use of an W = oo' analogue of the Markov chain C[^{s). 
The case N = oo does not make sense directly in terms of the original model Xt, but (A2) 
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can be used to define a Markov chain on the whole of which we can relate to our finite- 
Markov chains, at least when s < s*. 

We use Ct{s) to denote our new Markov chain, now defined on the whole of Z"*", and we 
write P for the associated probability measure and E for the corresponding expectation. The 
idea is to define transition probabilities via 

F[Ct+i{s) =m\Ct{s)=n]= lim Fn[C^+i{s) = m \ Cf (s) = n]; 

N^oo 

to show that this is legitimate under suitable assumptions, we need the following result. 
Lemma 6.2. Suppose that (A2) holds. Let s G [0, 1]. Then for any n,m & , 

p'{n,m) := lim Fn[C^+i{s) = m \ Cf (s) = n] 

is well-defined, and ^mizi+ P^in.m) = 1. 

Proof. We show that the increment distribution, conditional on {C^ = n}, given by (16.41) in 
the finite case converges (as — > oo), using assumption (A2), to an appropriate limiting 
distribution, which will serve as the increment distribution p*(n, ■). This convergence is 
clear for the term Bt+i{s), which has no iV-dependence. Moreover, given C^^ = n, the terms 
Bt+i{s) and y4f^^(n) are independent. Thus it suffices to show that y4^^(?7,) converges in 
distribution to a proper random variable. We show that this follows from (A2), although 
care is needed to correctly account for lost mass in (A2). 

To proceed, it is useful to introduce more notation. Let R = (i?i, . . . ,Rk) denote the 
N ^ oo distributional limit of R^: given (A2), this limit exists but is not necessarily a 
proper distribution on N^, but we recover a proper distribution by expanding the state- 
space to (NU {oo})^ . Thus components of R may take the value oo: this cannot be directly 
interpreted in terms of rank distributions, but is convenient for correctly accounting for the 
lost mass in (A2). Concretely, the distribution of R is given, for any k < K and any distinct 
zi,Z2, . . . ,Zfc e N, 

F[Ri = ii, . . . ,Rk = ik, Rk+i = oo, . . . , i?x = oo] 

= lim P^[< = ^i,...,< = ^,]- ^ ■■■Ei™ ^^[^1 

ifc+lSN ixSN 

= . . ,ik) - ^ ■■■^ K{ii,...,iK), 

using (A2). Note that since R^ is exchangeable on {1, . . . , A^}^', it follows that R is ex- 
changeable on (N U {oo})''^. 

Now we can define the N = oo analogue of Af_^^^{s) to be an independent copy of G 
{!,..., K} : Ri < n}, i.e., for R(t + 1) = {Ri(t + 1), . . . , Rxit + 1)) an independent copy of 
R, with distribution given by (16. 6p . we take 

K 

At+i{n) :=^l{i?,(t + l) <n}. 

i=l 



— ii, . . . , Rj^ — Ik] 

(6.6) 
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Then we can construct Ct{s) via its increments 

Ct+i{s) - Ct{s) = Bt+i{s) - At+^{Ct{s)). (6.7) 

Since converges in distribution to i? as — )■ oo, Af_^-^^{n) = J2f=i ^{^fi^ + ^ n} 
converges in distribution to ylj+i(n); specifically, using exchangeability, 

PAr[<i(n) = k] = (^^^^ Piv[< < n, . . . , < n, > n, ...,<> n] 

)P[i?i <n,...,Rk <n,Rk+i > n, . . . , Rk > n], 



as — oo. This completes the proof. □ 

The following result gives some basic properties of the Markov chain defined above. 

Lemma 6.3. Suppose that (A2) and (A3) hold. Then for any s G (0,1), Ct{s) is an 
irreducible, aperiodic Markov chain on Z"*", with uniformly hounded jumps: P[|Ci+i(s) — 
Ct{s)\ > K] = 0. The one-step mean drift of Ct{s) is given by 

E[a+i(s) - Ct{s) I Ct{s) = n] = K{s - G{n)). (6.8) 

Proof. The boundedness of the increments follows from the construction in (16. 7p . The irre- 
ducibility and aperiodicty follow from a similar argument to that used in the proof of Lemma 
I6.1l in the finite- iV case, now using (A3) in place of (Al). The drift (16.81) also follows similarly 
to the proof of (16.31) in Lemma 16. Ij in the present case 

n n 

E[At+i{n)] =J2^[te {Ru . . .,Rk}] = kJ2^[Ri = t], 

i=i i=i 

by exchangeability of R (see the comment after (16. 6p ). But ^1^=1 ^[^i = i] = G{n), by (A2) 
and the definition of G{n) at (13.21) . □ 



6.4 Large- asymptotics 

We show that properties of the Markov chains C^{s), described in Section [6^ in the large 
A^ limit can (at least when s < s*) be described using the W = oo' Markov chain Ct{s), 
described in Section 16.31 The main tool is Theorem 18.11 stated and proved in Section |H1 
Recall the definition of vr^^ from Lemma 16.11 

Lemma 6.4. Suppose that (Al), (A2), and (A3) hold, and that s G (0,s*). There exists a 
unique stationary distribution for Ct{s) , withn'^in) > for all n G Z+ vr''(n) = 

1, such that 

lim P[Ct(s) = n]= TT'{n), (6.9) 

for any initial distribution Co{s). In addition, the following results hold. 

(a) There exist c > and C < oo such that, for all n G Z^, t^nI^) — Ce~'^^ and TT^ln) < 
Ce"^". 
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(b) For any n G Z+, liiiiAr^^oo tt^I'^) = vr*(n). 

(c) Ast-^oo, E^[Cf (s)] En=o^^A^H nCt{s)] ^ Enez+ ^^'H < oo- 

Proof. We will show that we can apply Theorem 18. II with Y/^ = C^^(s), Yt = Ct{s), = 
{0, 1, . . . , N}, and 5 = 2+. Since s G (0, 1) and (Al) holds, Lemma EH] shows that C{^{s) is 
an irreducible Markov chain on {0, 1, . . . , N}, while, since (A2) and (A3) hold. Lemma [6.31 
implies that Ct{s) is an irreducible Markov chain on Lemmas 16.11 and 16.31 also imply 
that the increments of C^{s) and Ct{s) are uniformly bounded in absolute value (by K) 
almost surely. Thus (18. ip holds. 

Next we verify the drift conditions in (18. 2p . Since s < s*, there exists e > such that 
s < s* — 2e. First consider the finite- case. By (A2) and the definition of s* at f l3.3p . given 
e, we can take and no such that for any N > Nq and any n > uq, 

GN{n) > s* — e > s + e. 

So we have from (16. 3 p that, for all N > Nq and n > no, 

E,,[Cli,{s) - C^{s) I Cf{s) = n]< -sK. 

A similar argument holds for Ct{s), using (16. 8p . Thus (18. 2p is satisfied. Finally, we verify 
(18. 3 p by Lemma 16.21 Thus Theorem 18.11 applies, yielding the claimed results. □ 

The next result deals with the case s > s* . Recall that tn{s) defined by (14.71) denotes 
the time of the first return of {s) to 0. 

Lemma 6.5. Suppose that (Al), (A2), and (A3) hold, and that s > s* . Then 

limAr^ooEAr[rAr(s)] = OO. 

Proof. Suppose that s > s*. Then, for some s > 0, s — s* — e > s. Fix x G N. Since, by 
(A2), limAT^oo G7v(n) = G{n) < s* for any n, we can find ^"0(0;) such that GAr(n) < s* + s 
for any n < x and any A^ > Nq{x). Hence, by (16. 3p . 

E^[C,^,(s) - Cf (.) I (.) = n] > Ke, (6.10) 

for any A^ > Nq{x) and any n < x, where e > does not depend on x. We show that (I6.10p 
implies that C^{s) has a positive probability (uniform in x) of reaching x before returning 
to 0, which will imply the result. It suffices to suppose that Cq{s) > 1. 

To ease notation, write r := tn{s) for the remainder of this proof. To estimate the 
required hitting probability, set Wt := exp{— 5C/^(s)}, for 5 > to be chosen later. Now 

Wt^, -W, = exp{-5Cf (.)} (exp{-5(C,^i(s) - Cf (s))} - l) 
< exp{-5Cf (.)} {-6{C^^,{s) - Cf (.)) + , 

for some absolute constant M, using the fact that the increments of Cl^{s) are uniformly 
bounded. Taking expectations and using (16.101) . we have that, on {C^{s) < x}, 

EN[Wt+i - Wt I Cf (s)] < exp{-5Cf (s)} {-Ke6 + M5^) < 0, 
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for S < Sq small enough, where 6q > depends only on e and not on x or A^. Let 
z/j. := min{t G Z+ : C^{s) > x}. Then we have shown that Wt^rAu^ is a nonnegative 
supermartingale, which converges a.s. to WrAu^- It follows that 

so that PAr[^'a; < t] > 1 — e""^ =: p, where p > does not depend on x or on A^. The fact 
that Cj^(s) has increments of size at most K implies that on {u^ < r} we have {r > x/K}. 
Hence ¥n[t > x/K] > FnIi^x < t] > p, so that En[t] > px/K for all > No{x). Since x 
was arbitrary, the result follows. □ 



6.5 Proofs of Theorems SH and [3721 

Proof of Theorem \3.1[ First suppose that s < s*. Then Lemma 16.41 applies. By Lemma 
16.4( a). vr^(?T,) < Ce~'^^ where C < oo and c > do not depend on or n. In particular, 
for any p > 0, sup^J^nez+^^'^Ni^) < Moreover, by Lemma [6^ b). ii^^{n) — )■ 7r^{n) 
as A^ — 7- oo. Hence for any p > 0, by uniform integrability, Xln '^*'^Af('^) ~^ Sn'^*'^*('^) 
A^ — 7- oo. Together with Lemma [6.4( c). this implies that, for s < s*, 

lim lim En[C^(s)] = lim V mr%(n) = V mi'(n) < oo. (6.11) 

Next suppose that s > s* . Then, for fixed x > and some e > (not depending on x) we 
have that (16.101) holds for any A^ > Nq[x) and any n < x. On the other hand, if Cf^ {s) > x, 
we have that C^i{s) > x — K (by bounded jumps). It follows that 

E^[C^^,{s) - Cf (.) I (.)] > K{1 + e)l{C^{s) <x}-K. 

Taking expectations implies that 

^n[C^+i{s)] - (s)] > K{1 + e)¥^[C^{s) <x]-K. 

By (16.21) . the left-hand side of the last display tends to as t — ?■ oo. It follows that, for 
some 6 > that depends on e but not on x, Fi\f[C[^{s) > x] > 6 for all t large enough. 
Hence E7v[C/^(s)] > x6, for all A^ and t sufficiently large. Since x was arbitrary, and 6 did 
not depend on x, the second part of the theorem follows. □ 

Proof of Theorem \3.^ For s < s*, the statement follows immediately from Theorem 14.11 

Suppose that s > s* . For the duration of this proof, we write r for tm{s) to ease notation. 
In this case. Lemma [631 applies, showing that limAr^^oo lE^f [r] = oo. We claim that {s) is 
asymptotically null in the sense that, for any n G Z"*", 

lim lim P^[Cf (s) < n] = 0. (6.12) 

Indeed, (I6.12p follows from Lemma 16.51 and the occupation-time representation for the sta- 
tionary distribution of an irreducible, positive-recurrent Markov chain (see e.g. [H Corollary 
1.3.6, p. 14]) gives: 

lim P„|Cr(.) < „| = ±.U-) = ^"g-°;'f(W^ "' ; 



20 



in the final fraction, tlie denominator tends to infinity with (by Lemma 16. 5p while the 
numerator is uniformly bounded in since the expected number of visits to any bounded 
interval stays bounded, by irreducibility (uniform in A^). Thus fl6.12p holds for s > s*. 
Taking expectations in fl6.3p yields 

^n[C,1,{s)] - Ejv[Cf (s)] =Ks- KEm[Gm{C^{s))]. (6.13) 

The left-hand side of f l6.13p tends to as t — )■ oo by (16. 2p . Also, for uq as in (A4), 

lim lim E^[G7v(Cf (s))l{Cf (s) < no}] < lim lim Fn[C^{s) < uq], 

which is by (I6.12p . Hence, taking limits in (I6.13p . we obtain 

s = lim lim E7v[G7v(Cf (s))l{Cf (s) > ^o}]. 

By condition (A4), a.s., 

G^(Cf (s))l{Cf (.) > no} = s* + {l- s*)^^ + Bn, 

where Sn = o(l) as N ^ oo, uniformly in C^{s) (and hence uniformly in t). Thus 

s = lim lim (s* + (1 - s*)N"^EN[C^is)]) , 

N—^oo t—>-oo 

which yields the result (13. 9p for s > s*. 

Finally, suppose that s = s*. Then C^{s) < C^{r) for all t, N, and r > s*. Hence 

lim lim (N^^E[C[^{s)]) < lim lim lim (N~^E[C^ (r)]) 

= lim V{r), 

r4-s 

by the previous part of the proof, since r > s*. The latter limit is V{s*) = 0, so the result 
(13. 9p is proved for s = s* as well. □ 

7 Proofs for Section [4 
7.1 Overview 

In this section we work with the K = 2 case of Example (E3), working towards proofs of 
the results in Section |H The organization of the section broadly mirrors that of Section O 
In Sections 17.21 and 17.31 we return to the finite- A^ Markov chain C^{s) and the limit chain 
Ct{s), respectively, describing their properties more explicitly in this special case, for which 
exact computations are available. Then in Section 17.41 we give the proofs of our remaining 
results. Theorems 14.31 14.41 and 14.51 
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7.2 The Markov chain C^{s] 



For this model, the following result complements the general Lemma WAl Write p%{n, m) := 
Pjv[Ct^i(s) = m I Cf (s) = n]. Recall the definition of from (gl]), and that Fn{0) = 
Fjv(l) = 0. In the case where Fj^{n) = jP^, Vn^'"^^ ^) written down in equations (l)-(3) 



m 



Lemma 7.1. For any s G [0,1], (C/^(s))tgz+ is a Markov chain on {0, 1, 2, . . . , A^} under 
Pat. The transition probabilities are given by 



p^(0,0) = (l-s)2, p%{0,l)=2s{l-s), p%{0,2) = s 



2 



and for n > 1, 



p%{n,n-2) = {l-sYFN{n) 

p%{n,n- 1) = 2s(l - s)FN{n) + (1 - s)\l - 

p%in, n) = s^F^in) + 2s{l - - 
p%{n,n + l) = s\l-FN{n)). (7.1) 

Moreover, for n > 0, 

E^[C,^i(s) - Cf (s) I Cf (s) =n] = 2s-{l + F^in))l{n ^ 0}. (7.2) 

Proof. Suppose that C/^(s) = n. If n = 0, then the two points that we select come from 
(s, 1), and C^^{s) is 1 or 2 according to whether 1 or 2 of the new points land in [0, s]: each 
does so, independently, with probability s. This gives p^(0,m). 

Suppose that n > 1. In this case, X^-^^ < s is always removed. Then C/^^(s) is either 
(i) n — 2; (ii) n — 1; or (iii) n according to whether (i) the second point selected for removal 
is one of {xl'^\ . . . ,xj:^^}, and both new points fall in (s, 1); (ii) the second point is one of 
. . . , Xj"^}, and exactly one of the two new points falls in (s, 1), or the second point is 
one of {xj:^~^^\ . . . , X^^^}, and both new points fall in (s, 1); or (iii) the second point is one 
of {xl^^^\ . . . , X^^''}, and both new points fall in [0, s]. Thus we obtain the expressions in 
(17. ip . noting that the probability that one of . . . , X^"^} is selected as the second point 

for removal is -F/v(n). We then obtain (17. 2p from (17. ip . □ 



7.3 The 'iV = oo' chain Ct{s) 

Again we consider the W = oo' chain Ct{s) as described in Section lOl In the special case 
where (A2') holds, so that F is the limiting distribution given by (14. 2p . and a = 0, the 
transition probabilities p'^{n,m) = ¥[Ct+i{s) = m \ Ct{s) = n] are given for n = by 

p^(0,0) = (l-s)2, p^{0,l) = 2s{l-s), p^i0,2) = s', 

and for n G N by 

p''(?7.,?7. - 1) = (1 - s)^, p'{n,n) = 2s{l - s), p^n.n + I) = . 
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In their analysis, de Boer et al. [7] discuss this Markov chain, although they do not give 
full justification that it can be used to describe the asymptotics of the finite- chains C^{s); 
this is justified in a specific sense by our results from Section EiH which rely on the technical 
tools from Section [HI In the remainder of this section we present some basic properties of 

The Markov chain Ct{s) is almost a nearest-neighbour random walk (or birth-and-death 
chain), apart from the fact that from we can make a jump of size 2. However, the form of 
the transition probabilities allows us to use a trick to transform this into a nearest-neighbour 
process (see the proof of Lemma [7^ below). We will prove the following result, which corrects 
an error in the stationary distribution proposed in [7j. 

Lemma 7.2. Suppose that a = and s < 1/2. Then for any n G Z"^, 

limP[a(s) =n] =7r"(n), (7.3) 



and the stationary distribution tt'^ satisfies (^7^. Moreover, 

limE[a(s)] = 2s + -^. (7.4) 

Proof. Observe that the probability of a jump from to 2 is the same as that from 1 to 
2 (namely, s^), while the probability of a jump from into the set {0,1} is the same as 
that from 1 into {0, 1} (1 — s^). So we can merge {0, 1} into a single state and preserve the 
Markov property. (A formal verification of the preservation of the Markov property under 
this transformation is provided by, for example, [HI Corollary 1].) This gives a genuine birth- 
and-death chain on a state-space isomorphic to Z"*". Call this new state-space {0, 1, 2, . . .}, so 
that corresponds to {0, 1} and n for n > 1 corresponds to n-|- 1 in the original state-space. 
This new Markov chain has transition probabilities 

g''(0, 0) = 1 - s^, g*(0,l) = s^, and for n > 1, q'^in.rri) = p'^in + l,m + I). 

This Markov chain is reversible and solving the detailed balance equations (cf e.g. [6l §1.12, 
pp. 71-76]) we obtain the stationary distribution vf* for s < 1/2 as 

r(n) = 

for all n > 0. To obtain '/r'*(n), the stationary distribution for the original Markov chain, 
we need to disentangle the composite state 0. We have that vr'*(0) -|- vr''(l) = vr''(0) and, by 
stationarity, 

7r^(0) = (1 - s)V(O) + (1 - sfn'il). 

Solving these equations we obtain (14. 6p . Some algebra then yields the mean of the distribu- 
tion Ti^ (when s < 1/2), giving 

^ n7r^(n) = 2s + ^— ^ < oo, (7.5) 




since s < 1/2. Hence (I7.4p follows from Lemma [6.4( c). □ 
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7.4 Proofs of Theorems I4.3L I4.4L and 14.51 

Now we can complete the proofs of our remaining theorems. 

Proof of Theorem \4.3[ The s < 1/2 statement follows from (16. lip and (17. 5p . On the other 
hand, for any s > 1/2 and any r < 1/2, we have C^{s) > C^{r), so 

lim lim E^[Cf (s)] > lim (^2r + — — ^ = oo, 

N^oot-^oo rtl/2 \ 1 - 2r / 

as required. □ 



Proof of Theorem 4-4 ■ The s < 1/2 part of the theorem follows from Lemma UTI] and Lemma 
EMh). On the other hand, for any s > 1/2 and any r < 1/2, P^vlCf (s) < n] < P7v[<^f (r) < 
n] so that, again by Lemmas 17.21 and 16.4( b) . 

n 

lim lim Fn\C^(s) < n] < hm V 7r7m) = 0, 

m=0 

by gl]). □ 

Proof of Theorem \4.5{ Since F^lxj:'^^ < s] = ¥n[C[^{s) > n], we have 



n— 1 n— 1 



lim hm PtvI^^"^ < s] = 1 - hm lim V Pjv[Cf (s) = m] = 1 - V 

N-^oo t~^OD Af— i-oo t— >oo — ^ — ' 

r?i=0 r?i=0 

by Theorem 14. 4[ The result (14. 9 p now follows from (14. 6 p and some algebra. □ 



8 Appendix: Markov chain limits 

In relating the asymptotics of the Markov chains Cj^(s) to the W = oo' Markov chain Cj(s), 
we need the following general result (Theorem 18. ip on a form of analyticity for families 
of Markov chains that are uniformly ergodic in a certain sense (but not the sense used 
in Chapter 16 of [26], where the uniformity is over all possible starting states of a single 
Markov chain; our 'uniformity' is over a family of Markov chains all starting at the same 
point). Theorem 18.11 is related to material in Chapters 6 and 7 of [11], although our setting 
is somewhat different and the proof we give uses different ideas. Our context differs from 
the set-up in [TT], most notably in that our state-space changes with N, unlike in [TT]. It 
is likely that the methods of [11] could be adapted to our setting. However, it is simpler to 
proceed directly; we use, in part, a coupling approach. Recall that a subset 5 of M is locally 
finite if S" fl i? is finite for any bounded set R. 

Theorem 8.1. Fix Nq G N. For each integer N > Nq let Y/^ be an irreducible, aperiodic 
Markov chain under P^ on Sn a countable subset of [0, oo), where e Sn, Sn C Sn+i, and 
lim sup;\r^(,o S'at = oo. Also suppose that Yt is an irreducible, aperiodic Markov chain under 
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lim [qN{x, y)l{x G Sn}] = q{x, y), (8.3) 

A"— >oo 



P on S := UnSn- Suppose that S is locally finite. Write and E for expectation under 
Pjv and P respectively. Suppose that there exists B < oo such that for all N > Nq, 

P^[|r,^i - Yf\ >B] = 0, and P[\Yt+i - Yt\ > B] = 0. (8.1) 

Suppose also that there exist Aq G (0, oo) and > for which 

sup sup EnIY^I^ - Y/^ I F,^ = x]< -So; 

N>No a;g5ivn[Ao,oo) 

sup -Yt\Yt = x]< -So. (8.2) 

xGSn[Ao,oo) 

Let qN^x.y) := PN[Y^^-y = y \ Y/^ = x] and q{x,y) := P[Yt+i = y \ Yt = x]. Suppose that 

lim [qNix,y 

for all x,y & S . Then the following hold. 

(a) The Markov chain Yf is ergodic on S and, for any N > Nq, Y/^ is ergodic on Sjy. Let 
Tn '■= min{t G N : Y^^ = 0} denote the time of the first return to hy the process Y^^ ; 
similarly let r := min{t G N : = 0}. There exists 5 > such that 

E[e^^] < oo, and sup Piv[e^^^] < oo. (8.4) 

N>No 

(b) There exist stationary distributions on Sn and v on S such that 

lim Pn\YI^ = x] = vn{x), and lim P\Yt = x] = iy{x). 

t—^oo t—^oo 

Moreover, there exist c > and C < oo such that for all N > Nq and all x G Sn, 
^n{x) < Ce'""" and, for all x e S, u{x) < Ce'""^. 

(c) For any x E S, limTv-i-oo J^n{x) = ^{x). 

(d) Finally, 

lim En\Y^] = y xu^ix) < oo, and lim E[Yt] = > xiy{x) < oo. 

xGSn x£S 

Before getting into the details, we sketch the outline of the proof. The Foster-type 
condition (18. 2 p will enable us to conclude that the Markov chains have a uniform (in A^) 
ergodicity property implying parts (a) and (b). We then couple Yt and Y/^ on an interval 
[0, A] where A is chosen large enough so that the processes reach before leaving [0, A] with 
high probability. Given such an A, we choose large enough so that on this finite interval 
(18. 3 p ensures that the two Markov chains can, with high probability, be coupled until the time 
that they reach 0. This strategy, which succeeds with high probability, ensures that the two 
processes follow identical paths over an entire excursion; using the excursion-representation 
of the stationary distributions will yield part (c). 

An elementary but important consequence of the conditions of Theorem 18. II is a 'uniform 
irreducibility' property that we will use repeatedly in the proof; we state this property in the 
following result. Note that the condition (18. 3p is stronger than is necessary for parts (a) and 
(b) of Theorem 18.11 in the proof of Theorem 18.11 (a) and (b) below, we use only the uniform 
irreducibility property given in Lemma 18.11 
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Lemma 8.1. Under the conditions of Theorem \8.1\ for any A G (0, oo) there exist ei := 
ei{A) > 0, Ni{A) G N, andno{A) G N such that, for all N > Ni{A) and allx,y G SNn[0,A] 
there exists n := n{x,y) < no{A) for which 

p^[i;^ = y|yo^ = x]>£i. (8.5) 

Proof. Fix A G (0, oo). Local finiteness implies that S (1 [0, A] is finite. Take large enough 
so that Sn n [0, A] = S n [0,A]. Irreducibility of Yt implies that for any x,y E S, there 
exists n{x,y) < oo such that P[Yn(x,y) = U \ = > 0. There are only finitely many pairs 
x,y & S r][0, A], so for such x, y, in fact P[F„(^. y) = y \Yo = x] > e2{A) > where n{x, y) < 
uq^A) and €2 and Uq depend only on A, not on x,y. Moreover, for any x,y G S' fl [0, A], 
there are only finitely many paths of length at most no(^) from x to y. It follows that for 
any x,y E [0, A] we can find a sequence of states of S", xq = x,xi,. . . ,Xn-i,Xn = y with 
n = n{x,y) < no{A) for which 

P[Fi = xi, . . . , F„ = x„ I Fo = xo] > es{A) > 0. 
However, by fl8.3l) we have that, as — j- 00, 

PN[Yf = Xi,...,Y^ = Xn\YQ^ = xo] = qwixo, Xi)gAr(Xi, X2) ■ ■ ■ qN{Xn-l,Xn) 
q{xo,xi)q{xi,X2) ■ ■ ■g(x„_i,x„) = P[Yi = xi, . . . , F„ = a;„ | Fq = xq]. 

So for all N large enough, Pn[Y^ = y | F^f = > e3{A)/2, say. □ 

Now we move on to the proof of Theorem 18. 1[ The theorem and its proof are in parts 
closely related to existing results in the literature, in particular certain results from p]-[^[TT| 
[T8l[22l[25l[26l[30] amongst others. However, none of the existing results that we have seen fits 
exactly into the present context, and rather than try to adapt various parts of these existing 
results we give a largely self-contained proof. In the course of the proof we give some more 
details of how the arguments relate to existing results. 

Proof of Theorem \8.1\ (a) and (b). First we prove part (a). Since Y/^ is an irreducible, aperi- 
odic Markov chain on the finite or countably infinite state-space Sn, the drift condition (18. 2p 
enables us to apply Foster's criterion (see e.g. [11]) to conclude that Y/^ is positive-recurrent 
(ergodic) and in particular, since (18. 2p is uniform in A^, En[tn] is uniformly bounded (inde- 
pendently of N). 

In fact, we have the much stronger result (18. 4p . The exponential moments result (18. 4p for 
a specific is essentially a classical result, closely related to results in pT|l22p26] . for instance, 
and follows from the drift condition (18. 2p together with the bounded jumps condition (18. ip 
and irreducibility: concretely, one may use, for example. Theorem 2.3 of [18] or the a = case 
of Corollary 2 of [2]. The uniformity in (18. 4p follows from the fact that (18. 2 p and (18. ip hold 
uniformly in A^, and that the irreducibility is also uniform in the sense of Lemma ISTTl Indeed, 
the results of [21[TB] apply not to Tn itself but to := min{t G Z+ : Y/^ < Aq} where Aq is 
the constant in (18. 2p : standard arguments using the uniform irreducibility condition extend 
the uniform bound on E'^vfe'^'^^] to the desired uniform bound on i?7v[e^^^]. In particular, 
(18. 4 p implies that for any k eN there exists < 00 such that 

^[r'l < Ck, and sup Ejv[4] < Ck, (8.6) 

N>No 
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a fact that we will need later. This completes the proof of part (a). 

By positive-recurrence, there exist (unique) stationary distributions on and u on 
S such that limf_j.oo -P/v[^t^ = x] = Vn^x) and limf_j.oo P\Yt = x] = z/(x). Next we prove the 
uniform exponential decay of z/at and v. Again these results are closely related to existing 
results in the literature, such as those in [I8l[25l[30], Chapters 6 and 7 of |Tl], Section 2.2 
of [3], or Section 16.3 of [26] . 

For 5 G (0, 1), let Wt ■= e^^K We show that Wt has negative drift outside a finite interval, 
provided 5 > is small enough. We have that 



E[Wt+i -Wt\Yt = x]= e'^^'E[e^(^*+i-^') -l\Yt 



x\. 



Taylor's theorem with Lagrange remainder implies that for all y G [—B, B] and all 6 G (0, 1), 
e^y — 1 < 6y + KS^, where K := K{B) < oo. Using this inequality and the bounded jumps 
assumption (18. ip . we obtain 

E[Wt+, -Wt\Yt = x]< 6e'^ {E[Yt+^ -Yt\Yt = x]+ K6) 

< 5e^^ i-eo + K5) , 

when X > Aq, by (18.21) . Hence, for S := 6{B,eo) G (0, 1) sufficiently small, we have that 

E[Wt+i -Wt\Yt = x]<0, (8.7) 

for X > Aq. 

Let cr := min{t G Z+ : Yt < Aq} and z/^- •= min{t G Z+ : Yt > x}. By irreducibility, 
a A < oo a.s.. Moreover, by (18.71) . VFtAo-Av^ is a nonnegative supermartingale. Hence 
W^iAaA;.. W^Au^ a.s. as t OO, and 

e^^o = Wq> E[W^;,,J > E[W,l{a > z/J] > e''^P[a > z/J. 

The same argument holds for Wf^ := e*^^'^, uniformly in > A^^q. Thus we have 

P[u, <a\YQ = y]< e-'^^-y\ and P^K, < aj, \ Yq"" = y] < e-'i-y\ (8.8) 

where vn,x '■= min{t G Z"*" : > x} and ajy '■= min{t G Z"*" : Y/' < Aq}. 

We deduce from (18.81) . with the uniform irreducibility property described in Lemma [8. 
that the probability of reaching [x, oo) before returning to decays exponentially in x, 
uniformly in A^. We will show that 

P[jyx <r]< Ce-''^ and PN[jyN,x < tn] < Ce-^\ (8.9) 

By uniform irreducibility (Lemma 18. ip and the bounded jumps assumption (18. ip . we have 
that there exist Ai G {Aq, oo) and ^ > for which 

inf P\t < ua. \ Yq = y\> 6, and min inf Pn\tn < A^ I = vl > ^• 

Together with (18. 8p . this will yield the result (18. 9p : the idea is that each time the process 
enters [0, Aq\, it has uniformly positive probability of reaching before it exits [0, Ai], other- 
wise, by (18. 8p . starting from [Ai, Ai + B] the process reaches [x, oo) before its next return to 



27 



[0, A] with an exponentially small probability, and (18 .Op follows. We write out a more formal 
version of this idea for Yt only; a similar argument holds for Y/^ . 

Let Kq := and for n G Z+ define iteratively the stopping times ?7„ := min{t > : Yf > 
Ai} and Kn+i '■= min{t > rjn '■ Yt Aq} . By successively conditioning at these times (all of 
which are a.s. finite), we have 

PW. <r]< P[u, < K, I Y^,] + E [PK < -r I I ^o] 

< < '^n+i I ^o] + E [P[r/i < r I Y^,]E[P[v^ < r \ F^J | F«J | 
KCe-'"" {1 + (1-6) + (1-6)^ + ■■■), 

since P[?7„ < r | Y^^J <1~9 a.s., and P[u^ < k„+i | < Ce"*^^ by ( IHTSjl and the fact that 
Yr,„ <Ai + B a.s., by dUI]). Thus we verify flHlQj) . 

Let Lj^{x) denote the total occupation time of state x & hy Y/^ before time tn, i.e., 
during the first excursion of Y/'; similarly for L{x) with respect to Yt. That is, 

TjV-l r-1 

L^{x) := J2 = and L(a:) := ^ IjFt = x}. 

t=o t=o 

Standard theory for irreducible, positive-recurrent Markov chains (see e.g. [H Corollary 1.3.6, 
p. 14]) gives 

= aud = (8^10) 

By (18. 9p . the probability of visiting x during a single excursion decays exponentially. In 
order to bound the expected occupation time, we need an estimate for the probability of 
returning to x starting from x. We claim that there exists £2 > for which 

max max P/v [return to x before hitting I Yf^ = x] < 1 — eo, (8.11) 

N>No xeSNn[Ao,oo) ^ o I u J - ' V y 

and also max^.g5n[yio,oo) P[return to x before hitting \ Yq = x] < 1 — 62- To verify (I8.1ip . 
note that from ([hSD and ([HI]) there exists e' > such that P[Yt+i -Yt< -e' \Yt = x]>e' 
for all X > Aq, and the same for Yf (uniformly in A^). Then ([HSD yields (IgHT]) . It follows 
from (18.111) that, starting from x, the number of returns (before hitting 0) of Yt or Yj^ to x 
is stochastically dominated (uniformly in and x) by an exponential random variable. In 
particular, 

En[Ln{x)] < CP^vKx < tn], and E[L{x)] < CP[v, < r], 
which, with (I8.10p . yields the claimed tail bounds on uj\f and u. □ 

Proof of Theorem \8.1\ (c) and (d). First we prove part (c). We will again use the represent- 
ation ( I8.10p . We use a coupling argument to show that, as — )■ 00, for any x G S", 

En[Ln{x)] E\L{x)l and PjvM ^ E[t]. (8.12) 

Let e > 0. Take A G (0, 00) large enough so that BCi/A < e, where Ci is the constant in 
the k = 1 version of ( 18. 6 p and B is the bound in (18. ip . Also, for convenience, choose A so 
that A/ B is an integer. We claim that 



lim sup |g7v(x, ?/)!{?/ G S'tv} - g(x,?/) I = 0. (8.13) 

yes 



^^°°a;G5jvn[0,2A] 
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To see this, note that since 5" (and hence also Sn C S) is locally finite, in the supremum in 
f l8.13p . X takes only finitely many values (uniformly in A^), and, by f IS.ip . only finitely many 
terms in the sum are non-zero (again, uniformly in A^). Hence by condition fl8.3p we verify 
the claim f l8.13p . By fIS.lSp . we can choose large enough such that 

sup y2\qN{x,y)l{y e Sn} - qooix,y)\ < eB/A. (8.14) 

xeSjvn[o,2A] 

We couple Yf and Yt. We take large enough so that ^tvH [0, 2A + B] = Sn [0, 2A + B]. 
We use the notation for the probability measure on the space on which we are going to 
construct coupled instances of Y/' and Yf, and write for the corresponding expectation. 
We start at Y^^^ = Yq < A. We claim that one can construct (Y/^ , Yf) clS cL Markov chain 
under so that = y\ Y^^ = x]= q^ix, y), P^^lYt = y\Yt = x]= q{x, y), and 

P;^[(F,^,F,) = {y,y) \ (F,^,F,) = {x,x)] > min{g^(x, y), g(x, y)}. 

To see this, observe that when Y/^ = Yt = x we may choose transition probabilities rr^{y, z) = 
PNiO^t^^Yt) = {y,z) I {Yf,Yt) = {x,x)] satisfying r^{y,y) = mm{qN{x,y), q{x,y)}, 
J2z^y ^x{y, z) = qN^x, y), and J2yjtz ^x{y, z) = q{x, z): these constraints can always be satis- 
fied by some choice of r^- If Y^ 7^ Fj, we define by allowing the two processes to evolve 
independently. 

Let a := min{t G N : Y^ 7^ denote the time at which the processes first separate. 
For any t < A/B, we have from (18. ip that msix{Y/^ ,Yt} < 2A a.s., and together with the 
fact that the state-spaces of the two processes coincide on [0, + B], (I8.14p implies that, 
for t < A/B, P^fla >t + l|a>t]>l - {Be/ A). Hence, for any t < A/B, 

P;,[a>t]>l-(^-^y (8.15) 

Let Et denote the event Et := {a > t} H {tn < t}, i.e., that the paths of Yt and Yt^ 
coincide up until time t and visit zero by time t. Then, by f l8.15p . 

Pn[Ea/b] < PnW < A/B] + PnItn > A/B] < e + (BCi/A) < 2e, (8.16) 

using Markov's inequality and (18. 6p to bound P^[tn > A/B], and the choice of A to obtain 
the final inequality. On Et, {tjv = t}, so that 

E*^[\r^-T\]<E*^[\T^-T\1{E^^/^)] 

<{EUT'^]+E%[r']y/'{P;,[E^^/^]y/\ 

by the Cauchy-Schwarz inequality. By (I8.16P and the k = 2 case of (18. 6p . this last expression 
is bounded above by e^/^ times a constant not depending on A^. Since e > was arbitrary, 
the second statement in fl8.12p follows. 

Similarly, on Et, {L]\f{x) = L{x)} for any x G 5, so that 

EUlLNix) - Lix)\] < EUlL^ix) - L(x)|l(i?l/^)] 
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since Ljs[{x) < and L{x) < r a.s.. Thus we obtain the first statement in f l8.12p . Combining 
the two statements in fl8.12p with the representation in (18.101) we obtain i^Ar(x) — > z/(x) for 
any x E S, completing the proof of part (c). 

Finally we prove part (d). The convergence results follow from, for example, Theorem 2 
of [30] once the integrability of the stationary distributions is established. But the fact that 
Y^xujsrlx) and ^ xz/(x) are finite follows from the bounds in part (b). □ 
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